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Automatic detection and retrieval of embedded invisible digital watermarks from halftone 




images 





(57) A system for automatic detection and retrieval 
of embedded invisible digital watermarks retrieves dig- 
ital watermarks from halftone images. Specifically, by 
supplying an image to the system, through a process of 
autocorrelation and shifting, the embedded invisible wa- 
termark becomes visible. The process includes scan- 
ning or supplying an image to the system, cak;ulating 



the global autocorrelation of the image, selecting a mov- 
ing window size, conducting a piecewise localized au- 
tocorrelation for each window-sized portion of the im- 
age, retrieving the embedded, Initially invisible, water- 
marks, normalizing the resultant Image for visualization 
and displaying the resultant image with the now visible 
retrieved embedded digital watermarks. 
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great as the primary, or absolute, peak amount of transmitted light that occurs when the images are perfectly aligned 
However, this secondary relative peak is detectable. Therefore, if watermarks are embedded in a halftone image i e 
in the periodic structure of the image. autocorreIatk>n repeats itself to an extent when the periodic occurrences of the 
watermark are themselves aligned between the two images. 

[0011] Therefore, by using a two-step autocorrelation process, the system and method of this invention enables 
recovery of invisible digital watermarks from printed copies, 

[001 2] These and other features and advantages of this invention are described in or are apparent from the following 
detailed description of the prefen^ed embodiments. 

[001 3] The preferred embodiments of this invention will be described in detail, with reference to the foltowing figures. 



^0 wherein: 



Figure 1 is a halftone image containing an invisible or embedded digital watermark; 
Figure 2 shows the embedded watemnark retrieved by using the method according to this inventbn; 
Figure 3 is a functional block diagram of a watermark detection device according to this invention; 
Figure 4 is a functional block diagram showing the autocorrelator of Fig. 3 in greater detail; 
Figure 5 is a functional block diagram outlining in greater detail the global autocorrelation determiner of Fig 4- 
Figure 6 is a functional block diagram outlining in greater detail the piecewlse autocorrelatbn determiner of Fig' 4' 
Figure 7 is a flowchart outlining the watemnark retrieval process according to this invention; 
Figure 8 is a flowchart outlining in greater detail the global autocorrelatbn determination step of Fig. 7; 
Figure 9 is a flowchart outlining in greater detail the piecewise autocorrelation determination step of Fig. 7; 
Figure 10 shows retrieved digital watermarks using a first scanning resolution; 
Figure 11 shows retrieved digital watenDart<s using a coarse scanning resolution; and 
Figure 12 shows watermarks retrieved from the image shown on Fig. 1 using a constant shift. 

[001 4] Fig. 1 shows an image 1 00 containing a hkiden. or embedded, digital watermark. Fig. 2 shows a processed 
image 100', formed by processing the image 1 00 according to the method of this invention, to retrieve or extract the 
embedded watermark 110. The image 100 was selected because the image 100 reflects one of the most difficult 
situations for watermark retrieval. Specifically, the image 100 shown in Fig. 1 is an irmge of a chapel, created by a 
180x90 dpi stochastic halftone screen with an embedded invisible digital watermark, printed at 400 dpi on a black and 
white pnnter The embedded digital watermark is an 'X" logo. This particular stochastc screen is designed so that the 
left 90x90 pixel square is identical to the right 90x90 pixel square, except within the area specified by the shape of the 
X logo of the embedded watermark. Within the X logo, all corresponding pairs between the left and right squares are 
conjugates. 

[0015] In the image 100' shown in Fig. 2. the presence and clarity of the embedded digital watermark 110 retrieved 
from the onginal watermark-encoded image 100 of Fig. 1 can be seen, presuming the image has remained in unaltered 
digital fomnat since the watemiark embedding process. Operating in the digital realm, if the watermark^mbedded 
digital image 100 shown in Fig. 1 is copied, the original image and the copy can be digitally overlayed The copy is 
then digitally shifted 90 pixels to the right relative to the original image. When the two images are then togically ANDed 
together, the embedded watemnark 110 becomes clearly visible, as shown in the image 100' shown in Fig. 2. 
[0016] Thus, by using the method and system of this invention, retrieving watennarks from images that have not 
been outside the digital realm since the embedding process is straightforward and produces clear results However, 
retneving the same watermarks after the image has been transferred to a printed copy is not trivial. 
[0017] However, it shoukJ be appreciated that this image is not limited in any way to a printed image The only 
requirement tor retrieving an embedded digital waternnark is that the image on which the retrieval operation is performed 
was once encoded with an embedded digital watermarit. The system and method of this invention works equally well 
on images that have been printed and subsequently scanned, that have been previously converted into a digital image 
or that have been maintained in electronic form. 

[0018] Fig. 3 shows a watermark extraction device 200 for extracting embedded digital watermarks from an image 
according to this invention. As shown in Fig. 3, an image containing an embedded digital watermark is input from an 
image input devk:e 300 over a link 31 0 to the watennark extractk>n device 200. It should be appreciated that the image 
input device 300 can be any devrce that stores and/or generates an electronic version of the image. 
[0019] Thus, the image can be a printed hardcopy version of the image, and the image input device 300 can be a 
^nner that scans and outputs an electronic version of the image over the link 31 0 to the watermark extraction device 
Furthermore, the scanner 300 and the watemoaric extraction device 200 can be elements integrated into a diqrtal pho- 
^ tocopier. 

[0020] Similarty. the image input device 300 can be a server or other node on a k>cal area network a wide area 
network, an intranet, the Internet or any other distributed network. In this case, the image is already stored on the 
network in electronic form. Finally, the link 31 0 can be a wired or wireless link to the scanner or other image conversion 
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244. a watermark retrfej; ^I. ::rri::r„'J™^ amocorrelafor, deterrr,iner 243. a shifted image generator 

no. .he memo^ 230 an. o.p^lZrj^e'^Xt^^^^^^^^^ ^ 

selector 242 moves over the input image and outputs a plurality of poSLToTt^e fnml Lln» ^ 

spondingiothesizeofthemovingwindowandtoacurremLittrortheSnoI^^^^ """"^ 
determiner 243 inputs the plurality of portions and ompJ^Kf S al^^ZZforirr^^^^^^^ autocorrelation 

[0024] 



30 



3S 



40 



[0026] Fig, 6 shows the piecewise autocorrelation determiner 243 of Fio 4 in nrpator h,»*o,i tk 

relation detem,iner 243 includes a moving window positioner lisfan in^'ageXer 2^2 a 1^.^'^*"" ^"^^ 

a mean subtracter 2434 a local autorrtrr^^iatinn nit^rr^i^ ^T^J ^ cropper 2432, a mean determiner 2433, 

determiner 243 begins by movino the selected winriou^ a^roTeT^ . ^' P'^cewise autocorrelator 

the peak locator 2436 locates a local D^ak for th J f.L that portion of the image. Then. 

[0028, Simi^H, the output device 400 can be a server or other node on a iL area network, a wide area network 
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an intranet, the Internet or any other distributed network. In this case, the resulting image is transferred and stored on 
the network in electronic form. Finally, the link 410 can be a wired or wireless link to the output devce 400 or any other 
image output or display device or to the network. Thus, the output device 400 and the link 410 can be any known 
elements that are capable of receiving and outputting or storing the resulting electronic image from the watermark 
5 extraction device 200. 

[0029] Fig. 7 outlines a method according to this invention for retrieving embedded watermarks from images that 
have been converted to a printed copy. Beginning in step S1000, control continues to step S1100, where the printed 
copy image is scanned. It should be appreciated that, if the image is already in electronic format, control jumps directly 
to step SI 200. 

10 [0030] In step SI 200, the image is analyzed to determine if the image is a halftone image and to estimate a global 
autocorrelation for the image. Next, In step SI 300, a moving window is selected. The size of the moving window is 
based on the estimation of the global autocorrelation analysis. Then, the image is f urtheranalyzed by iteratively applying 
the nrioving window over the entire image. At the beginning of each iteratran, in step SI 400, a next moving-window- 
sized portion of the image is selected. Then in step S1500, the piecewise localized autocorrelation for each selected, 

IS moving window-sized portbn of the image is determined. Control then continues to step SI 600. 

[0031] In step S1600, based on the results of the k>calized autocorrelation determination, an estimate of the kx^al 
peak is determined for each selected portion of the image, A shifted image will be generated for each selected portion 
of the Image based on the peak value of the kx:alized autocorrelation detenmined for that selected portion. Next, in 
step SI 700, the embedded watermark is retrieved. Then, in step SI 800, the data containing the selected portion of 

20 the image with the retrieved watermarks is normalized and stored into the memory for later visualization. In step SI 900 
the control routine determines if the entire image fias been selected and analyzed. If not, control jumps back to step 
SI 400. Otherwise, if the entire inr^age has been analyzed, control continues to step S2000. In step S2000, the control 
routine stops. 

[0032] The resolution of the input device does not need to match the resolution of the input printed copy innage. 
2S Importantly, the resolution of the input device can be lower than the resolution of the printed copy image. As discussed 
below, the system and method of this invention are capable of successfully detecting watermarks from images that 
were printed at 400dpi and scanned at SOOdpL Furthermore, if the resolution of the input device used to scan the printed 
copy image increases, the signal-to-noise ratio increases and the contrast of the retrieval watermark in the resulting 
image is enhanced. 

30 [0033] Due to the periodicity of the halftoning process, the global autocorrelation of a halftone image determined in 
step SI 200 presents peak values at certain positions. If the image has rennained in unaltered digital format since the 
watermark embedding process, these autocorrelatbn peaks are located exactly as a two dimensional comb function. 
For example, the halftone image 1 00 shown in Fig. 1 was generated by a stochastic halftone screen with the periodicity 
of 90 pixels in both x and y direction. Therefore, the autocorrelation peaks of innage 1 00 are shown as a two dimensional 

35 comb function with 90 pixel separation in both the x and y directions. To determine the existence of this comb function 
and its periodicity and orientation, autocorrelation peaks other than the one at (0, 0) position are searched for. For the 
example image shown in Fig. 1, two autocorrelation peaks at (90, 0) and (-90, 0) are located on the horizontal axis 
and two peaks at (0, 90) and (0. -90) on the vertical axis. 

[0034] However, if the halftone image has been converted into hardcopy fomnat, i.e. , is moved out of the digital realm, 

40 the printing and/or copying process, as well as the scanning process to convert the halftone Image back to the digital 
realm, may introduce unkrxjwn scaling, rotation, distortion and noise into the reformed digital image. For example, the 
halftone image 100 shown in Fig. 1 was printed by a black and white printer at 400 dpi and scanned by a scanner also 
at 400 dpi. Theoretically, four peaks of the autocorrelation function on the horizontal and the vertical axes should remain 
at (90, 0), (-90, 0), (0, 90) and (-90, 0) locations. When searching for the actual global autocorrelation, two correlation 

45 peaks are located near the horizontal axis at (89, 1) and (-89, 1), and two peaks near the vertical axis at (-1 , 90) and 
(1, 90). Therefore, if the embedded watermarks are assumed to have been generated by a stochastic screen with a 
horizontal arrangement, as described above, searching for localized correlation peaks by the piecewise autocorrelation 
determiner 243 can be reduced to searching only around point (89, 1) of each 90x90 pixel portbn of the image. 
[0035] It shoukJ be appreciated that this search process can be straightforwardly varied to encompass embedded 

50 watermarks that have a vertical orientation or even a plurality of orientations. Furthemrjore, the system and method of 
this invention encompasses retrieving digital watermarks from color irrages. By using the same halftone screen for 
color prints, i.e., for each color separation layer of a CMYK image, detecting the emb>edded watermarks in a color 
image is performed identically to the process outlined above. However, if a different halftone screen is used for each 
color separation layer, the retrieval process must be performed independently on each cotor separation layer 

55 [0036] The moving window used in step SI 300 should be large enough to cover an area containing a portion of two 
adjacent correlated hafftone pattems. For example, for the image 100 shown in Fig. 1 , the moving window could vary 
from as smalt as 100x20 pixels to as large as 256x256 pixels. Larger moving windows provide a higher signal-to-noise 
ratb and faster speed in the kx:alized piecewise autocorrelation determinatkwi performed in step SI 500. In contrast. 



5 



BNSOOCID: <EP_0961175A2J_> 



EP0 951 175 A2 



10 



15 



Gres(M) = Gs^„(i.j).G(i,j). 

where: 

g"'''^ t r^'^K*!!"^ ''^^^ "^^^^ watermarks are visible 

^shifi (I. J) IS the shifted image data at original location (i, j); and 

G (i, j) is the original image data at original location (i. j). 

measure of the absolute value If tha imaoo « n^t = Lih^ average value of their neighborhoods in relation to a 
tostepS1900 If Inst^ SiS) theT/nfillf ^^^^^^^^ where control is returned 

steo Si9^Jcir ^ ^^220 the image is detenroned to be a halftone image, control continues to step S1230 In 
stepS1230. the s.ze and or.ntat«>n of the ha«one image is estimated. Next, in step S1 240. contro. is rlmJi^o^teJ 

wndow-sized portion of the image was selected in step S14(r^rpT,ticut? S s eo Sl^^^ cument moving- 

isrml^ con,™, ^JS^S^'i^^"' >"« m«,.ub,««« 

P<».«„ o, ,h, ,oca, ^J. peat ™, ^ir^^SS'^^J^tTC.^ S^l 
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x«c = ip^ 5I«(ip+1.jp) - f(ip-1,jpM2-f(ip.j,) - f(ip+i j^)- f(ip.i,j^)]. 
yacc-jp-^-5f(ipjp+l).f(i^j^.1)y[2.f(ipJpH(ipJp+1)-f(ip.jp-1)J. 



where: 
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f(i, j) is the calculated autocorrelation functbn for the current portion (i. j), 
OnJn) is P63k position within the current portion, and 

(^acc. Vacc) 's an estiniation of an accurate maxinaal position within the cun-ent portion. However, it should be ap- 
preciated that there are other methods of perfornning this interpolation. 

[0044] In step S1 600. with the estinnated peak position by step S1 500, the shifted version of the scanned image can 
be generated by: 

Gshift(M) = Wi-G(i + int(X3^).j + int(y3^)) + w^G(\ + intCx^^) + 1, 
i + '"^(yaoc)) + ^3-00 + int(X3^).j + intfy^^) + 1) + 
w^.G(i +int(x3^) + l.j + intCy^^) + 1). 



Wi = {1 .0 - [x^^ - int(X3^]}.{1 +(yacc-'nt(yacc)l). 

W2 = lXacc-'nt(Xaoc)l (1 " [yacc-'"t(ya(xl}= 
20 W3 = {1 .0-[X3^-int(X3^]}.[y3^-int(y3^]. 

W4 = [Xacc-'nt(Xaoc)l-[yacc-int(yacc)] and 
G(i, j) is the input image. 

The shift values x^^^^ and y^^^ are determined from the piecewise autocorrelation determination for the corresponding 

25 current portion, where both (i, j) and (\+x^cc^ j+Yacc) covered, 

[0045] Figs. 10 and 1 1 show examples of embedded watermark retrieval as outlined above in Figs. 5-7. The above- 
outlined method was conducted on a halftone image, printed by a 400 dpi printer and scanned in both 300 and 400 
dpi modes. Fig. 10 illustrates the image 500 resulting from performing the method of this invention on a 400 dpi printed 
inrrage that was scanned at 400 dpi. The recovered watenrarks 510 are clearly visible. Fig, 11 illustrates the image 

30 600 resulting from performing the method of this invention on a 400 dpi printed image that was scanned at 300 dpi. 
The recovered watermarks 610 are also clearly visible. 

[0046] By comparison, as shown in Fig.12. efforts to retrieve watermarics by applying a constant shift detemiined by 
the gtobal autocorrelation to the entire image 100 resulted in an image 700, whrch was both printed and scanned at 
400 dpi. Note the X logo watermarks 71 0 are only cleariy visible on the left hand portion of the image shown in Fig.l 2. 

35 [0047] As shown in Fig, 3, the watermark extractk>n device 200 is preferably implemented on a programmed general 
purpose computer. However, the watermark extraction device 200 can also be implemented on a special purpose 
computer, a programmed microprocessor or microcontroller and peripheral integrated circuit elements, an ASIC or 
other integrated circuit, a digital signal processor, a hardwired electronic or logk; circuit such as a discrete element 
circuit, a programmable togic device such as a PLD. PLA, FPGA or PAL, or the like. In general, any device, capable 

40 of implementing a finite state machine that is in turn capable of implementing the flowcharts shown in Figs. 5-7, can 
be used to implement the watermark extractbn device 200. 
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Claims 

1. A method for recovering an embedded invisible digital watermark from an image, comprising: 



determining a global autocorrelation of the image; 
selecting a window having a determined size; 
so selecting a current portion of the image corresponding to the size of the window, 

determining a piecewise localized autocorrelation of the image for the current portion; 
generating a shifted innage for the current port ran; 
retrieving the embedded watermarks for the current portion; 

repeating the portion selecting, determining, generating and retrieving steps for a new current porton until the 
^ entire image has been analyzed; and 

generating a new image containing the retrieved watermarks. 

2. The method of claim 1 , wherein detemnining the global autocorrelation comprises: 
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searching the image for autocorrelation peaks; 
determining if the image is a hatftone; and 
estimating a size and orientation of the halftone. 

^' letermf^wlf Z "^'7 ^' '^^^^'l ^'^^ comprises determining the window size, the 
SaeTnfnhT '"^ "^T '° determining the window size, the determined window size being 

large enough to ensure a portion of two adjacent correlated halftone patterns is contained within the window 

l^rZ!""^ °' ^"^ °' ^'^""^ ^ '° ^' ^^'^^'"9 P°^'°" °* •l^® '"^^ corresponding to the size of the 

the image at the current position to the size of the window. "HP">g 

^p'lJs^r °' ^"^ °* ' '° d«'«""'*9 the localized amocorrelation for the current position 

detennining a mean of the current portion; 
subtracting the mean from the current portion; 
determining a local autocorrelation of the current portion; 

locating a local peak of the current portion near a point estimated by the global autocorrelation' and 
determining a maximal correlation between the Image and the shifted image for the current portion. 

6. The method of any of claims 1 to 5, wherein generating the shifted image comprises: 

GshfflC'i) = w, .G(l + int(x^^),j + int(y^^)) + ^N^.G,(\ + ■mt(x^) + i . 

j + '"'(Vacc)) + W3-Q(' + int(X3„).j + '"t(yaec) +1) + 

w^GCi + int(x^) + 1 ,j + int{y^) + ^ ). 

where: 

Wi = 11 0 - Kcc - int(X3„)} {[1 - intl),^])- 
W2 = [x^ - int(x,^)].{[l iy^ . int(ya^)]}; 
W3 = {1 .0 - [x^ - int(X3<J} .[y3„ - in\{y^l 

W4 = [Xacc- 'nt(Xace)] -[Vaec ' 'nt(yaee)]: and 
G(i.j) is the Input image; and 

shift values x and y^^ are determined from the piecewise autocorrelation determination for the correspond- 
ing current portion, where both (i. j) and (i+Xa„, j+y,^) are covered. 

7. A recovery system that retrieves embedded invisible digital watemiarks from an Image comprising: 

Input means for inputting the image; 

global autocorrelation means for detennnining a global autocorrelation for the image- 

ol meTmagT*^''^'^''^" "'^^^ ^ "'^^"^^^ autocorrelation for each of a plurality of portions 

;S::^|':ut'^^^^^^ ^'"'^ ^'^ °' "'"'^•"^ ^'^^ °' '-9^ ''-^ - the 

v3^TS=i:i:?=^^^ 

output means for displaying the new image containing the retrieved watermark. 

8. The system of ciaim 7. wherein the global autocorrelation means further comprises: 

searching means for searching the image for peaks; 
determining means for determining if the image is a halftone; and 
estimating means for estimating a size and orientation of the halftone. 
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9. A system that extracts a digital watermark from an irrage based on global and piecewise autocorrelation compris- 
ing: 

a controller; 
5 a memory; and 

an autocorrelator. 

10. The system of claim 9. wherein the autocorrelator comprises: 

a global autocorrelator that determines a gfobal autocorrelatbn of the image; 
a moving window selector that selects a moving window; 

a piecewise autocorrelator that determines a piecewise local autocorrelation of the image for each of a plurality 
of portions of the image; 

a shifted image generator that generates a shifted version for each portion based on the corresponding piece- 
^5 wise autocorrelation; 

a waternoark retriever that retrieves a watermark for each portion based on that portion and the corresponding 
shifted image; and 

an image visualizer that generates, based on the retrieved watermarks, a new image containing the retrieved 
visible watermarks. 
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